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ABSTRACT 



Colorado Senate Bill 00-71 and S.B. 00-124 established the 
Read- to- Achieve Grant Program. The purpose of the program is to solicit 
proposals from any elementary school, including a charter school, or 
consortium of schools, to fund research-based intensive reading programs. The 
funding opportunity was designed specifically for second and third grade 
students and students between third and fourth grades whose literacy and 
reading comprehension skills are below the level established by the State 
Board of Education in the Colorado Basic Literacy Act (CBLA) . In fiscal year 
2000/2001, the program received a total appropriation from the Tobacco 
Settlement Funds of $15 , 895 , 687- - 99% of the funds have been distributed 
directly to schools implementing intensive reading programs through 
Read-to-Achieve grants. One of the main principles of the program is to 
ensure accountability for the planned and actual use of the funds. A 
comprehensive review process involving 222 trained reviewers from across 
Colorado with expertise in reading and school -based reform was used to make 
sure research-based programs were in place in funded schools. Detailed 
feedback was provided to all 817 applicants for these funds. This annual 
report consists of the following: Executive Summary; Background; Part 1: 
Implementation of Grant Program; and Part 2: Evaluation of Program. (Includes 
2 tables. Contains three attachments: Rules for the Administration of the 
Read-to-Achieve Program; Read-to-Achieve Board; and Sample Feedback 
Document . ) (NKA) 
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Executive Summary 



During the 2000/2001 school year, the Read-to-Achieve Grant program successfully 
accomplished the goals established in the legislation creating the program (22-7-506 
C.R.S.). All statutory requirements were addressed, rules were established, and local 
grants were distributed. The Read-to-Achieve Board provided oversight for all activities 
of the program. The Colorado Department of Education Competitive Grants Unit 
administered the program. 

The purpose of the program is to solicit proposals from any elementary school, including 
a charter school, or consortium of schools, to fund research-based intensive reading 
programs. The funding opportunity was designed specifically for second and third grade 
students and students between third and fourth grades whose literacy and reading 
comprehension skills are below the level established by the State Board of Education in 
the Colorado Basic Literacy Act (CBLA). 

In FY 2000/2001, the Read-to-Achieve Grant program received a total appropriation from 
the Tobacco Settlement Funds of $1 5,045,687. Ninety-nine percent of the funds have 
been distributed directly to schools implementing intensive reading programs through 
Read-to-Achieve grants ($14,895,687). One percent of the funds, as stipulated by 
statute, were used for administrative costs, including training and support for grant 
applicants, external evaluation, and ongoing support and networking of grant recipients. 

A small amount of the administrative allocation ($16,954.60) was unspent. 

During FY 2000/2001, the Read-to-Achieve Grant program awarded grants to 550 
schools across the state (out of 817 submissions) providing services to almost 28,000 
students. This represents funding for 75% of the students at grades 2 and 3 who are on 
Individualized Literacy Plans (ILPs). Data confirm that distribution closely parallels both 
the overall student population across the state and numbers of students on Individual 
Literacy Plans (ILPs). 

A comprehensive review process involving 222 trained reviewers from across Colorado 
with expertise in reading and school-based reform was used to make sure research- 
based programs were in place in funded schools. Detailed feedback was provided to all 
817 applicants for these funds. Colorado Department of Education staff provided 
extensive support in planning and implementing intensive reading instruction through 
regional trainings, web-based support, timely consultation, and step-by-step guides. 

One of the main principles of the Read-to-Achieve Grants program is to ensure 
accountability for the planned and actual use of the funds. The Department has 
contracted an external evaluator to implement a comprehensive evaluation that 
addresses each of the statutory requirements for evaluation and reporting, provides clear 
expectations for grantees, works within a parameter of using no more than 1% of the 
budgeted time from each grant, and provides support structures for completing evaluation 
requirements. 
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This report is submitted to the Colorado Department of Public Health and Environment to 
detail progress made in implementing the Read-to-Achieve Grants program from July 1 , 
2000 through June 30, 2001. The report is divided into three sections: background, 
program implementation, and program evaluation. 




Colorado Read-to-Achieve Grant Program Annual Report 

8 



5 



Background 



Senate Bill 00-71 and S.B. 00-124 established the Read-to-Achieve Grant program. The 
resulting legislation enacted by the General Assembly is 22-7-506 Colorado Revised 
Statutes (C.R.S.). 

The Tobacco Litigation Settlement Cash Fund provides an ongoing source of funds for 
the program. By statute, 19 percent of the Settlement Cash Fund is appropriated 
annually to the Read-to-Achieve Cash Fund, with a maximum annual appropriation of 
$19 million. In addition, S.B. 00-124 included a FY 1999/2000 appropriation of $7.0 
million from the General Fund. Table 1 shows the actual and anticipated funding for the 
program for the next several years. Ninety-nine percent of the funds have been 
distributed directly to schools implementing intensive reading programs through Read-to- 
Achieve grants. One percent of the funds, as stipulated by statute, were retained for 
administrative costs, including training and support for grant applicants, external 
evaluation, and ongoing support and networking of grant recipients. A small amount of 
the administrative allocation ($16,954.60) was unspent. 



Purpose of Program 

The purpose of the program is to solicit proposals from any elementary school, including 
a charter school, or consortium of schools, to fund research-based intensive reading 
programs. The funding opportunity was designed specifically for second and third grade 
students and students between third and fourth grades whose literacy and reading 
comprehension skills are below the level established by the State Board of Education in 
the Colorado Basic Literacy Act (CBLA). Funded activities can include reading 
academies for intensive reading instruction, after-school literacy programs, summer 
school clinics, tutoring, and extended-day reading programs. 

Rules for Administering Grant Program 

The State Board of Education is responsible for promulgating rules for the grant, 
including application procedures, criteria for selecting schools and determining grant 
amounts, and processes to evaluate the success of the programs operated by grant 
recipients. See Attachment A for a copy of the Rules for Administration. The Department 
of Education administers the grant. Please note: The State Board Rules default to the 
criteria within the Read-to-Achieve statute and the Request for Proposal (RFP) available 
on the Colorado Department of Education website at 
http://www.cde.state.co.us/cdecomp/r2a app.htm . 

Each elementary school applying for funds has addressed specific expectations within 
the scoring guide including the requirement that 25% of the students involved in the 
program meet the reading standard within the first granting period. All of the 
requirements for receiving these dollars relate directly to the expectations of the Colorado 
Basic Literacy Act (CBLA) - 22-7-501 through 22-7-505 Colorado Revised Statutes 
(C.R.S.) 
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TABLE 1 SUMMARY - READ TO ACHIEVE FUNDED GRANTS 
Round 1 and Round 2 
Budget & Cash Projections 



! i 


YEAR 1-Round 1 
Jan. 1, 2001 - 
June 30, 2002 
(18 Months) 


YEAR 1 -Round 2 
July 1,2001 - 
June 30, 2002 
(12 Months) 


YEAR 2 
July 1,2002 - 
June 30, 2003 
(12 Months) 


YEAR 3 
July 1 , 2003 - 
June 30, 2004 
(12 Months) 


TOTAL 






Number of Grants 415 138 1 




Approved or Projected Grants 
Approved-Round 1 
Approved-Round 2 
Projected-Round 1 
Projected-Round 2 


$26,964,195 


6,548,575 


$16,997,570 

4,456,000 


$15,967,778 

4,865,870 


$26,964,195 

$6,548,575 

$32,965,348 

$9,321,870 


Total App. or Proi. Grants 










$75,799,988 






Appropriation Dollars Available 
FY 2000-01 & FY 2001-02 
FY 2002-03 
FY 2003-04 


$36,264,787 




$19,958,952 


$19,958,952 




Total Dollars Available 


; ■ . t . J 








$76,182,691 



Available Balance 



1 $9,300,592| $2,752,0171 $1 .257,3991 $382,7031 $382,7031 



Fiscal Years 2000-01 and 2001-02 appropriation years are used to fund Round 1 and Round 2. 



Fiscal Year 2000-01 Appropriation* $18,849,309 ' ^ • ' 

Fiscal Year 2001-02 Appropriation** $17,415,478 

Fiscal Year 2002-03 Appropriation**’ $19,958,952 , . . , •• 

Fiscal Year 2003-04 Appropriation**’ ”• $1 9,958,952 

Total . $76,182,697] • ' " 

' .... + / _ * . , i . * , ’ ■ * 

*FY 2000-01 includes $14,895,687 Tobacco Revenue (reduced by $149,990 administrative funds) and $3,953,622 Cash Funds 
**FY 2001-02 includes $16,275,0.03 Tobacco Revenue (reduced by $164,394 administrative funds) and $1,140,475 Cash Funds 
***FY 2002-03 includes $19,000,000 Tobacco Revenue and $058,952 Cash Funds . ■ 

****FY 2003-04 includes $j 9,000,000 Tobacco Revenue and $958,952 Cash Funds ' ' ' . * ' ’ 



PER PUPIL AMOUNTS 



Round 1 



Total Students 


21,011 


Minimum 


$115 


Maximum 


$1,650 


Average 


$1,284 
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Round 2 


Total Students 


6,873 


Minimum 


$352 


Maximum 


$1,100 


Average 


$934 



; - t- ; 



11/1/2001 



Role of Read-to-Achieve Board 

The program is administered under the direction of the Read-to-Achieve Board, which 
consists of 11 members representing education at both the state and local levels, both 
houses of the General Assembly, and parents of children who may participate in the 
program. See Attachment B for a listing of Board members. To meet the legislative 
intent of the Read-to-Achieve Grant program (22-7-506 C.R.S.), the Board in partnership 
with the Colorado Department of Education is responsible for the following goals: 

Goal 1 : Provide additional intensive reading services to all second and third graders on 
Individual Literacy Plans (ILPs) so that they will be proficient readers by the end 
of third grade. 

Goal 2: Collect and review applications for Read-to-Achieve grants. 

Goal 3: Recommend to the State Board of Education the schools that should receive 
grants as well as the duration and amount of each grant. 

Goal 4: Determine continued funding of grants based on adequate progress during 
granting period, e.g., grantee meets the goals established in the grant 
application including demonstration that at least 25% of the students enrolled for 
the prior year met the reading standard. 

Goal 5: Report to the Governor and to the General Assembly on the effectiveness of the 
program by February 1, 2004. 
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Parti: 



Implementation of Grant Program 



Distribution of Grant Dollars 

The Read-to-Achieve Board began meeting in August 2000 to develop a Request for 
Proposal (RFP) and scoring rubric for distribution of dollars to local schools that met the 
specifications of the legislation. By September 2000, the application information was 
distributed to all local school districts and made available on the Colorado Department of 
Education website at http://www.cde.state.co.us/cdecomp/r2a_app.htm. 

The first round of grant applications for Read-to-Achieve dollars was received by the 
Department of Education in November 2000. Schools were allowed to request funding 
for up to three periods. The first funding period covered 18 months, from January 2001 
through June 2002, and the remaining two periods covered subsequent fiscal years 
(2003 and 2004). The Department received 605 applications and awarded grants to 414 
schools in January 2001 to provide services to almost 21 ,000 students. In total, over $26 
million was granted out of more than $36 million available for the period January 2001 
through June 2002, with a statewide average grant per student of $1 ,284. A listing of 
Round 1 grant recipients is available on the website. 

The second round of grant applications for Read-to-Achieve funding was received by the 
Department of Education in May 2001 . Schools were allowed to request funding for up to 
three periods. The first funding period for these grants covered a 12-month period from 
June 2001 through June 2002. The Department received 212 applications and awarded 
grants to 138 schools. In Round 2, the statewide average grant per student was $934. A 
listing of Round 2 grant recipients is available on the website. 

In total, Read-to-Achieve has awarded grants to 550 schools across the state providing 
services to almost 28,000 students. This represents funding for 75% of the students at 
grades 2 and 3 who are on Individualized Literacy Plans (ILPs). 



Support for Local Applicants 

During Round 1 (Fall 2000), all schools applying for Read-to-Achieve grants had access 
to regional grant training sessions (held in 8 locations across the state). Training 
modules were also available on the CDE website. In addition, applicants had access to a 
"just in time" consultant contracted by the Department to provide support in the planning 
of these applications as well as feedback (based on the scoring rubrics) on drafts of 
applications. This support was well received by local educators. 

In Round 2 (Spring 2001), two additional supports were added. Examples are available 
on the Read-to-Achieve website of sections from grant applications that were scored at a 
highly fundable level. Another means of assistance available on the website and at each 
regional training session was the Guide to Writing a Successful Application. This step- 
by-step guide was developed primarily to meet the needs of small, rural schools that 
were less successful during the first round. The Guide has been well received by 
applicants across the state. 
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Grant Review Process 

Grant applications during the past year were reviewed through a comprehensive review 
process involving 222 trained reviewers from across Colorado. The three-person review 
teams consisted of superintendents, principals, teachers, and others with expertise in 
reading and school-based reform. Each reviewer participated in the same training 
available to applicants and in scoring trials prior to the actual review. To further assure 
inter-rater reliability, each team read and scored three grants in common. Statistical 
calculations were performed to ensure validity and reliability of scores. 

Each application was required to meet a quality standard of 49 points (out of 74 total 
points). The teams determined the applications meeting the quality standard and 
provided recommended funding amounts. The Read-to-Achieve Board made final 
recommendations to the State Board for funding. 

The format and content of the Read-to-Achieve applications closely parallel the 
expectations for other grant opportunities available through state and federal funding 
administered by the Department of Education. Each grant uses an explicit scoring guide 
or rubric to guide reviewers. Components related to needs assessment, student 
achievement goals, action planning, budgeting, and sustainability include wording and 
content that are consistent across funding sources. This consistency is intended to assist 
local schools in completing grant expectations in a focused, timely, and effective manner. 

Regional Distribution 

The following table shows the regional distribution of grant funds. The Statute requires 
the Read-to-Achieve Board to ensure, to the extent possible, that grants are awarded to 
schools in a variety of geographic areas of the State. The Board worked to assure that at 
least 50% of those requesting funds from each region were funded. Data from Round 1 
show that funding distribution closely parallels both the overall student population across 
the state and numbers of students on Individual Literacy Plans (ILPs). Round 2 data 
continued this equitable distribution pattern. 



Table 2. 



Distribution of Read-to-Achieve Funds Amona Education Reaions 
First Funding Period (January 2001 - June 2002) 




Grant Awards 


Elidible Students 


Reqion 


Amount 


Percent 


Number 


Percent 


Metro 


$15,904,000 


59% 


20,467 


55% 


North Central 


$2,910,000 \ 


11% 


4,897 


13% 


Northeast 


$394,000 


1% 


529 


2% 


Northwest 


$1,738,000 


6% 


1,667 


4% 


Pikes Peak 


$2,728,000 


10% 


4,591 


12% 


Southeast 


$1,923,000 


7% 


2,650 


7% 


Southwest 


$426,000 


2% 


1,202 


3% 


West Central 


$941,000 


4% 


1,573 


4% 


Total 


^ $26,964,000 


100% 


37,576 


100% 
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Feedback Regarding Ongoing Support and Networking of Schools 
A brief summary of ongoing activities and trainings provided for Read-to-Achieve 
participants documents the support and networking available to schools as well as 
feedback regarding effectiveness. 

September 2000 - Reading Summit 

School personnel from around the state came together before the application process. 
Participants learned about research-based practices, heard Barbara Taylor, a national 
expert on instructional practices that make a difference in literacy, and gained new 
information on successful strategies being used in schools across Colorado. There were 
11 breakout sessions available that discussed essential elements for implementing 
successful reading intervention programs (e.g., leveraging resources, using data) and 
best practices in literacy instruction. Participants consistently praised the new 
information they received and the perspectives around best practices. 

September 2000 - Grant Writing Training 

The Colorado Department of Education provided grant writing training in 8 regions of the 
state. Locations wre Pueblo, Niwot, Arvada, Pagosa Springs, Montrose, Eagle, Haxtun 
and Colorado Springs. In total, 545 participants attended. The average participant rating 
on the workshop content standards was 4.68 on a 5-point scale. Participants indicated a 
thorough understanding of the intent and purpose of the grant as well as application 
expectations. After the training, participants had access to “just in time” consultants who 
answered questions and reviewed grant proposals to help ensure quality based on 
scoring rubrics for each section of the application. 

November 2000 - Reader Training 

The 138 reviewers (46 teams) for Read-to-Achieve grant applications were comprised of 
superintendents, principals, teachers and other experts in literacy and school based 
reform. Training included a walk-through of the application and scoring rubric, focus on 
issues schools encounter in completing applications, hands-on practice scoring (building 
inter-rater reliability), as well as discussion of bias, conflict of interest, and confidentiality 
requirements. Reader feedback included primarily 4 or 5 ratings on a 5 point scale. 
Readers stated that using the rubrics to score helped insure consistency. Comments 
indicated that the process worked well and was fair. A number of readers also pointed 
out new clarity of understanding related to the purpose, parameters, and context of the 
Read-to-Achieve grant program. 

November 2000 - Read-to-Achieve Reader Review 

The 138 trained reviewers from around the state came together to review 605 
applications and make recommendations to the Read-to-Achieve Board on funding 
decisions. Reviewers spent the day in teams of three to review applications. Many 
readers commented that the training provided was exceedingly beneficial to their work. 
These readers also noted that reading a variety of grants from the various schools would 
help them when they applied for future grants. One request by readers was for exemplar 
grants they could review as examples of quality work. 
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April 2001 - Grant Writing Training for Round 2 

Grant training was provided in 8 locations around the state. A step-by-step guide was 
used in the grant writing workshops. Participants consistently rated the sessions highly 
and often pointed out how much they appreciated the specific concrete information that 
was presented. One participant commented, “Training sub-divided requirements of a 
grant into a comfortable attainable task ...” Another said, “This presentation dug deeper 
and provided clarification to an already well-written document. The additional support will 
be beneficial when writing the grant.” A total of 264 participants attended, representing 
163 schools. 

April 2001 - Regional Networking Days 

Between April 2 and 12, Networking Days took place in each of the 8 regions of the state. 
Participants from Round 1 funded Read-to-Achieve schools were able to spend time 
together talking about their program successes and challenges. Participants indicated 
the Read-to-Achieve monies were generally being used for: summer school programs, 
intensive tutoring, mentoring, literacy labs, literacy coaches, reading specialists, parent 
training, staff development, reading clinics, guided reading materials and take-home 
books. 

A number of different programs are being used to meet schools' individual goals and 
reading needs. Some examples of these programs are: SOAR to Success, Accelerated 
Reading, Guided Reading Approaches, Learning Network, LindaMood Bell, Open Court, 
Accelerated Reader, Scholastic Reading Counts, Fast Forward, Tactile Visual Phonics, 
HOSTS, Carbo, and National Literacy Coalition. 

Each school is focusing on specific needs of their students based on ongoing 
assessment. Following are some of the assessments schools are using: Developmental 
Reading Assessment (DRA), Individualized Reading Inventories (IRA), Star Assessment 
and Northwest Evaluation Association (NWEA) Levels Tests. 

By April 2001 , most schools had begun teacher training, purchasing materials and 
planning and implementing intensive programs. Many schools reported significant early 
progress being made within their school buildings. 

May 2001 - Reader Training 

Training was consistent with Round 1 Training. In addition, readers were trained using 
the step-by-step guide and exemplars used by applicants. A number of Round 1 readers 
participated during the second round. These readers were re-trained and included in trial 
scoring practices. Overall, participants rated the reader training at a score of 4.5 out of 5 
on the evaluation of content standards. Many commented that they appreciated the 
ongoing support. Another aspect that was highly rated was that of practice in scoring 
grants to help provide additional consistency among readers. 

May 2001 - Reader Review 

Eighty-four reviewers gathered to assess 212 Read-to-Achieve grant applications and 
make funding recommendations to the Read-to-Achieve Board. The groups were highly 
complimentary concerning having exemplar grants available during the process for 
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comparison. They also appreciated the added measure to help ensure inter-rater 
reliability and consistency. 

September 2001 Networking Davs 

Two networking days were held in September 2001, one in Denver and another in 
Glenwood Springs The networking day is a requirement of the Read-to-Achieve grant 
with the intent to provide information on best practices, identify issues in implementation, 
and support networking of the participating schools. All 550 schools were represented. 

At these sessions, each participant was given a copy of the Report of the National 
Reading Panel: Teaching Children. This is an evidence-based assessment of the 
scientific research literature on reading and its implications for reading instruction. 

Governor Owens provided the opening address. He was followed by national literacy 
expert Phyllis Hunter. A number of breakout sessions were available to participants. 
Topics included innovative practices for implementing the six dimensions of reading, 
noteworthy student achievement gains related to Read-to-Achieve efforts, and practical 
approaches to using data to drive change. A time was set aside for participants to 
network within regional groups about successes and concerns regarding implementation 
of intensive reading strategies. 

There were approximately 850 participants at the Denver site and an additional 150 in 
Glenwood Springs. Highly rated components of the full day workshop were: keynote 
speakers, evaluation sessions, budget sessions, regional networking and breakout 
sessions. Participants wanted additional information dealing with English Language 
Learners (ELL). 

Ongoing Role of Read-to-Achieve Board 

The Read-to-Achieve Board has been instrumental in guiding all activities around the 
grant process. The Board was effective in assisting with the process of creating the grant 
application. The design of the Board allowed principals and teachers from both rural and 
urban settings as well as literacy experts, parents and legislators to have input into the 
document. Members of the Board attended the regional training sessions and 
participated as reviewers in the grant reading process. After grants were reviewed the 
Board met to examine recommendations for funding from the teams. They also 
completed additional review on consortium grants and reviewed appeals. Five appeals 
were received; one was granted. 

When the Sounding Board around evaluation was created a Board member served on 
this committee to give input and bring information back to the Board. Board members 
also served on the committee to plan the Networking Day in September. One member of 
the Board presented a session on research-based literacy strategies during the 
Networking Day that was the highest rated session of the day. 

The Board has meetings planned this year for the purpose of organizing next year's 
Networking Day, creating decision rules for continued funding based on external 
evaluation, and making funding decisions. 
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Part 2: 



Evaluation of Grant Program 



Accountability 

One of the main principles of the Read-to-Achieve program is to ensure accountability for 
the planned and actual use of the funds. By establishing a competitive grant process, 
policymakers emphasized the value to distributing funds to schools with well-designed 
programs that were focused on accomplishing specified objectives. Accountability for 
Read-to-Achieve grants has been addressed through a rigorous application and 
evaluation process and the reporting of program outcomes. The grant applications are 
scored based on information provided by the schools regarding: 

• Student needs 

• Program goals and evaluation mechanisms 

• Planned use of funds to support students and teachers 

• Indicators of success 

• Budget cost-effectiveness and the ability to leverage other funds 

To ensure that the programs achieve intended results, future funding is conditional on 
schools showing progress in their reading programs. By statute, schools awarded grants 
in the first period are eligible for funding in subsequent years if they achieve the goals set 
forth in their applications and demonstrate that a minimum of 25% of the pupils enrolled 
in the program in the prior year improved their reading skills. 



Research Base for Read-to-Achieve 

The accountability process for the Read-to-Achieve program revolves around Colorado 
Student Assessment Program (CSAP) data, the Colorado Basic Literacy Act (CBLA), and 
the research based on the six dimensions of reading documented in the Report of the 
National Reading Panel - Teaching Children to Read (2000). 

The Colorado Student Assessment Program (CSAP) is a state assessment program 
designed to measure student achievement in relationship to the Colorado Model Content 
Standards. These standards are expectations specifying what students should know at 
particular points in their education. Assessment of reading occurs from grade 3 through 
grade 10. These results are used by the Read-to-Achieve program in assessing 
adequate progress related to 25% of students meeting the reading standard. 

In Spring of 1996, the Colorado General Assembly passed House Bill 96-1139, 
Colorado’s Basic Literacy Act (22-7-501 through 506 C.R.S.). The preamble to this Act 
states: 

It is the intent of the General Assembly that, after third grade, no pupil may be 
placed at a grade level or other level of schooling that requires literacy skills not 
yet acquired by the pupil. 

This Act mandates that all students will be reading on the third grade level by the end of 
third grade and before they can move on to a fourth grade reading class. This Act 
requires that the reading growth of all students be monitored carefully from kindergarten 
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through third grade. Those students not reading on that grade level will be placed on 
Individual Literacy Plans (ILPs), which are developed with the school and the family. 

District responsibilities include: 

1 . Assessing the reading performance of all students 

2. Placing students on ILPs if students are not reading on grade level 

3. Reporting to the state: 

a. The number and percentage of pupils in the third grade who read at or 
are above their grade level. 

b. The number and percentage of pupils enrolled in the district who are on 
ILPs. 

c. The number and percentage of pupils who have increased their literacy 
and reading comprehension levels by two or more grades during one 
year of instruction. 

Best practices in literacy must serve as a foundation for all literacy work. All Read-to- 
Achieve grantees developed proposals based upon principles in the Report of the 
National Reading Panel - Teaching Children to Read which includes the 6 dimensions of 
reading, phonemic awareness, phonics, fluency, building background and vocabulary, 
comprehension, and motivation. After a comprehensive needs assessment, each grant 
had to show how the school planned to implement each of the 6 dimensions. Schools 
developed achievement goals and action plans. The goals schools set had to be SMART 
(specific, measurable, attainable, research-based and time-phased). 

External Evaluation 

The statute requires the Read-to-Achieve program to report to the Governor and the 
General Assembly by February 1, 2004: 

1 . The number of schools that received grants under the program and the average 
amount of the grants; 

2. The number of students enrolled in intensive literacy programs funded by the 
program, the number of pupils enrolled who improved their reading skills to grade 
level or achieved proficiency on the state assessment in reading for their grade 
level in the year after starting the intensive literacy program, and the percentage of 
students who achieved proficiency on the state assessment for reading for their 
grade level in both the year after starting the intensive literacy program and the 
following year; and 

3. Whether any statutory changes are recommended, including but not limited to the 
appropriateness of the requirements for adequate progress. 

The Department has contracted an external evaluator to implement a comprehensive 
evaluation that will address each of the statutory requirements. The use of an external 
evaluator avoids conflict of interest and assures necessary accountability. The evaluator 
has worked closely with the Read-to-Achieve Board to design an evaluation that allows 
the Board 

(1) to make decisions about continued funding for the 550 schools, and 

(2) to evaluate the overall effectiveness of programs used across the state. 
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Board decisions regarding continued funding are scheduled for June 2002 (the end of the 
first grant period). 

The Read-to-Achieve evaluation focuses primarily on the following questions: 

• How well did schools achieve the grant-specified achievement goals (25% 
improvement standard)? 

• How well did schools achieve their other stated goals? 

• What program characteristics describe those schools that did attain the 
achievement goals and those that did not? 

Participants must submit five individual reports to meet evaluation requirements: 

• Program Profile 

• Implementation Survey 

• Survey of Schools’ Program Goals 

• Achievement Data Tables 

• Executive Summary/Suggestions for Others 

Following is a timeline and brief description of the External Evaluation reports- 
2001 

September 26 or 28, 2001 : Schools receive evaluation information at Networking 
Meetings 

September 26 or 28, 2001 : Schools complete Program Profile Forms (To 
highlight the important aspects of each program and provide data to help identify 
aspects of successful programs). 

The Program Profile includes: Program Goals and Objectives, Program 
Structures, Instructional Strategies, Student Experiences, and Program Context 
and Support. This information will be reprinted and broadly shared, with 
information about the variety of Read-to-Achieve programs being implemented. 



2002 

May 15 (Early Decision Schedule) First opportunity for schools to submit 
materials for Board review and funding decisions regarding adequate progress 
during first grant period. 

^ une 1 1 (Regular Decision Schedule) - Second opportunity for schools to submit 
materials for Board review and funding decisions regarding adequate progress 
during the first grant period. Materials received by CDE for funding decisions 
include: 

• Implementation Survey — To provide information about program 
implementation and context for understanding program success. 

• Survey of Schools’ Program Goals - To provide needed information 
about attainment of each school’s goals related to student achievement, 
professional development, and parent involvement as stated in the school’s 
proposal. 

• Achievement Data Tables - These tables present student information by 
grade level group (including assessment results, student characteristics, 
and program participation data) and complete the counts and calculations 
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needed for each participating grade level. These data will provide essential 
information about student achievement for program reviews related to 
funding and for subsequent analyses related to evaluation of program 
success. 

September 15 

• Executive Summary - A brief 2-page report summarizing programs and 
results for the Summary of Read-to-Achieve programs 2001/2002. These 
reports will be broadly disseminated. 

• Suggestions for Others - A one page summary. Suggestions focus on 
identifying student or school needs, program goals and their evaluation, 
program components, and general suggestions. This will enable other 
schools and their students to benefit from program experiences of the 
Read-to-Achieve participants. 

The parameter was set that each school would need to spend the equivalent of time 
equaling no more than 1% of the grant award on evaluation reporting. For example, for a 
$50,000 grant, this would equal 1-2 days. 

Accountability - Sounding Board 

To help produce an evaluation document that would fit the needs of the variety of Read- 
to-Achieve schools, a Sounding Board was created. The Sounding Board consisted of 
practitioners throughout the state who would be working directly with the Read-to- 
Achieve evaluation. The group was comprised of principals, teachers, assessment 
coordinators and grant coordinators. Each member brought a different perspective to the 
process. The Sounding Board met regularly throughout the summer 2001 to provide 
input and reaction to Read-to-Achieve evaluation documents. The group then 
participated in distribution and training of the materials during the September Networking 
Day. 

Results of State Auditor's Review 

During the 2000/2001 school year, programs within the Department of Education were 
reviewed by the State Auditor's Office. This performance audit of the Department was 
conducted under the authority of Section 2-3-103, C.R.S., which authorizes the State 
Auditor to conduct audits of all departments, institutions and agencies of state 
government. Read-to-Achieve was one of four CDE programs audited during this period. 
The audit work, which included gathering information through interviews, reviewing 
documents, and analyzing data, was performed between October 2000 and April 2001. 
Highlights of the recommendations and responses contained in the report provide 
additional evaluation data. 

1 . The audit pointed out that larger schools tended to be more successful in obtaining 
grant funds than smaller schools. Schools with more than 600 students were most 
successful in obtaining grants, while schools with 200 or fewer students were less 
successful. 
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In order to address this issue, meetings have been held with rural superintendents to 
identify additional means of addressing their needs during future grant distribution 
(especially targeted at districts with fewer than 300 students). A new process for 
reviewing schools with under 200 students is being implemented with the Department’s 
latest grant (Teacher Development). The meetings with the rural superintendents have 
been highly productive. The audit recommendation of designating a certain amount of 
funding for the category of small rural districts is a primary option being considered. 

Assistance (primarily targeted at small rural districts) during the second round of Read-to- 
Achieve grants included (a) a step-by-step guide to writing these grants and examples of 
successful grant components from the last grant review (from both small rural and urban 
applications) available at all of the 8 regional trainings and on the CDE website, (b) 
continued access to timely consultation in planning and writing the application, and (c) 
regional assistance through a focus on regional center consortium grant option as well as 
CDE regional team assistance. 

Since February 2001, regular meetings have been held with rural superintendents to 
address their concerns regarding the data showing that fewer schools with student 
populations below 200 applied and fewer were successful. Many of the solutions above 
were derived from these interactions. In addition, the Department is establishing an 
Advisory Council of superintendents of small rural districts from each of the eight regions 
to meet quarterly to provide advise regarding necessary supports and procedures for 
distributing grants. 

The Board and CDE are assuring that the Read-to-Achieve funding efforts are leveraged 
effectively with other funding sources for making sure students read by the end of third 
grade, e.g., federal Comprehensive School Reform Demonstration (CSRD), Colorado 
Reading Excellence Act (CREA) and Title I School Improvement grants plus expected 
increases in federal funds for reading grants. State level - Teacher Development grants. 

CDE will also track schools that have not benefited from Read-to-Achieve funds by the 
beginning of the 2001-2002 school year (a) to make sure they have access to upcoming 
federal and state grant options and (b) to provide additional support through regional 
teams and Title I assistance. 

The Read-to-Achieve Board and CDE will continue to focus on fair distribution of funds 
across regions in future funding cycles. 

2. Another issue addressed during the audit was that because the grant process is 
competitive, funding is not distributed to all students who are eligible. Costs for 
administering the comprehensive grant process were also cited. A formula 
approach was suggested. 

The Department disagreed with a formula approach on the basis of extensive research 
regarding formula-driven dollars through the federal Title I program. A majority of these 
funds have been used to fund literacy over a 20-year period. The critique regarding this 
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program is that dollars have not produced results for students. Data from the program 
have shown the need to provide accountability on both the front end of funding as well as 
through analysis of student results. Providing dollars to schools that do not have the 
leadership or organizational awareness in place for effective intervention, professional 
development, and parent involvement via a formula-driven mechanism has not worked. 

Finally, research on the time necessary to produce change in student performance would 
support continuation of current funding during the next three years to schools that have 
met the quality expectations of the grant review. These schools across the state have 
been notified by the Read-to-Achieve Board that they were approved for three-year 
programs contingent on adequate progress. 

In regard to costs, the Department is able to provide the comprehensive grant program 
activities, including training, support, review, and evaluation, for a cost within the 1% 
guideline. 

3. The Audit suggested the Department of Education improve the application process 
for consortium applicants in the Read-to-Achieve program by developing a clear 
definition of a consortium and allowing schools within a consortium to submit one 
joint application rather than individual proposals. 

CDE and the Read-to-Achieve Board will clarify the definition of consortium, follow the 
current 20 consortia applications closely, implement a combined consortium application 
for the state Teacher Development grants (November 2001) and use this same 
procedure with any future Read-to-Achieve fund distribution,. They will also explore the 
best mechanism for assuring that essential building analysis and planning occurs within 
the combined framework as this applies to future awards. Individual schools will still 
need to report results on meeting the 25% student performance expectations to qualify 
for continued funding. 

4. The audit report also indicated the Department of Education should improve its 
communication with Read-to-Achieve applicants. 

Although individualized feedback was provided to each of the 817 applicants, CDE and 
the Board will continue to work toward a more effective feedback process. During the 
second round of Read-to-Achieve funding, feedback was further refined to assure: 

• Adequate clarity regarding program issues 

• Clear representation of any budget issues 

• Consistency of feedback across all applications 

• Comments that assist next steps 

See Attachment C for a sample feedback document. 

5. Lastly, the audit committee recommended that the Department of Education 
should provide additional standardization of procedures to include an appeals 
process. 
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CDE and the Board have added an additional process for aligning team results. Each of 
the teams scoring grants during the second round of Read-to-Achieve grant distribution 
scored three identical applications. The external evaluator adjusted scores for each of 
the team’s grants using a statistical procedure for alignment based on these results. 

CDE and the Board included information concerning the appeals procedure for grants 
within each letter to unsuccessful applicants during the second round of distribution. 

Review of Goal Accomplishments 

Goal 1 : Provide additional intensive reading services to all second and third graders on 
Individual Literacy Plans (ILPs) so that they will be proficient readers by the end 
of third grade. 

• In progress. At present, 75% of the students on ILPs are receiving 
additional intensive reading services as a result of these grants. The 
rigorous application and evaluation process assures that effective 
research-based practices are in place for these students. Additional 
support is being provided to assist those schools not presently receiving 
grant funds. 

Goal 2 : Collect and review applications for Read-to-Achieve grants. 

• Accomplished. The comprehensive process described in this report 
details the expectations, evaluation rubric, training, support, review 
process, and individualized feedback included in the review. 

Goal 3: Recommend to the State Board of Education the schools that should receive 
grants as well as the duration and amount of each grant. 

• Accomplished. Recommendations to the Board during the 2000/2001 
school year included funding for 550 of schools (75% of the student 
population on ILPs) fora total of $14,895,687 of Tobacco Revenue from 
the Fiscal Year 2000/2001 Appropriation. Grants were funded for an 
initial 18-month period and for two additional years contingent on 
adequate progress. Table 1 on page 7 details the projected funding over 
the three year period. 

Goal 4: Determine continued funding of grants based on adequate progress during 

granting period, e.g., grantee met the goals established in the grant application 
including demonstration that at least 25% of the students enrolled for the prior 
year met the reading standard. 

• In progress. The evaluation design and instruments are in place. All 
grantees have received training in how to complete the evaluation 
expectations. Additional support is available from November 2001 
through May 2002 to make sure each school completes the reporting 
requirements. Decisions regarding continued funding are scheduled for 
June 2002. 
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Goal 5 : Report to the Governor and to the General Assembly on the effectiveness of the 
program by February 1 , 2004. 

• Pending. Data required to report to the Governor and General Assembly 
are being collected and analyzed by the external evaluator and CDE staff. 
This Colorado Department of Public Health and Environment annual 
report includes initial information on implementation and evaluation 
activities completed to date. In June 2002, data on local grant 
performance over the initial 18-month period will be submitted. Additional 
local data will be submitted Fall 2002. By November 2002, the second 
report to the Colorado Department of Public Health and Environment will 
be submitted. By February 2003, the external evaluator will complete a 
comprehensive evaluation of the effectiveness of the Read-to-Achieve 
program during the initial 18-month period. By February 2004, the report 
to the Governor and General Assembly will be completed. 



Read-to-Achieve Board Commitments/Concerns 

Three key issues remain a concern for the Board 

• Meeting the three-year expectations for funding the schools in the Read-to- 
Achieve program. The Board intends to use the dollars available from the 
Tobacco settlement funds through the 2003-2004 school year to meet the Board's 
commitment to fund Read-to-Achieve schools that are making adequate progress. 
Providing the time for schools to make significant changes in programming and 
student performance is essential. 

• Implementing a timely and thorough evaluation and review process by June 2002 
based on the design developed by the external evaluator. Effective 
implementation of this design will result in clear decisions by the Board about 
continued funding and appropriate feedback to all 550 participating schools. 

• Making sure technical assistance for planning intensive instruction for reading 
achievement is easily accessible to those schools that need it. The Board will 
work with CDE staff on the Regional Education Teams as well as Regional Title I 
Consultants to make sure effective assistance reaches schools not presently 
benefiting from these funds. 



Summary of Primary Accomplishments 2000/2001 

• Implemented the largest school-based grant effort ever undertaken in Colorado. 
Provided regional technical assistance and support across the state. Over 800 
elementary schools applied for these funds. 550 of these schools (75% of the 
student population on ILPs) were successful. 
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• Used a comprehensive review process involving 222 trained reviewers from 
across Colorado with expertise in reading and school-based reform to make sure 
research-based programs were in place in funded schools. 

• Provided detailed feedback to all 817 applicants for these funds. 

• Provided extensive support in planning and implementing intensive reading 
instruction through regional trainings, web-based support, timely consultation, and 
step-by-step guides. 

• Began work with superintendents from small rural districts to make sure the Read- 
to-Achieve funds are readily accessible for their student populations. 

• Developed an external evaluation model that answers the essential questions 
required by legislation, provides clear expectations for grantees, works within a 
parameter of using no more than 1% of the budgeted time from each grant, and 
provides support structures for completing evaluation requirements. 
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Attachment A 



* 



Colorado State Board of Education 
Department of Education 



1 Colorado Code of Regulations 301-47 



Adopted: October 5, 2000 

Amended: 

Attorney General Opinion: October 13, 2000 

Authority: Article IX, Section 1. Colorado Constitution. 22-2-1 06(l)(a) and (c); 22-2- 
107(l)(c); 22-7-409(1.5); 22-7-601 et seq.; and 22-7-506, Colorado Revised Statutes 
(C.R.S.) 



RULES 

i 

FOR THE ADMINISTRATION OF 
THE READ TO ACHIEVE GRANT PROGRAM 

2207-R-1.00 Statement of Basis and Purpose. 

The statutory basis for these Rules adopted October 5, 2000, is found in 
22-2-1 06(1 )(a) and (c), State Board Duties; 22-2-1 07(l)(c), State Board 
Powers; and 22-7-506, the Read to Achieve Grant Program, C.R.S. 

The Read to Achieve Grant Program, 22-7-506, C.R.S. . requires the State 
Board of Education to promulgate rules which include, but are not limited 
to: application procedures by which a school may apply for grant funds 
through this program, criteria for selecting those schools that shall receive 
grants, the criteria for determining the amount to be granted to the selected 
schools, and procedures for reviewing the success of the intensive literacy 
programs operated by schools that receive grants. 

2207-R-2.00 Definitions. 



2.01 ( 1 ) 



Read to Achieve Grant Program. 

A grant program created in the Department of Education 
pursuant to 22-7-506, C.R.S. , allowing any public school to 
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apply for grants to fund intensive, research-based reading 
programs for second and third grade pupils and pupils 
between third and fourth grades whose literacy and reading 
comprehension skills are below the level established by the 
State Board of Education for pupils at each grade level. 

2.01 (2) Read to Achieve Board. 

An advisory board to the State Board of Education created 
pursuant to 22-7-506 (2)(a) and (3)(a), C.R.S. The 
advisory board’s membership and terms of office are 
defined in 22-7-506 (2)(a), C.R.S. The Read to Achieve 
Board’s major duties include the solicitation and review of 
applications for grants under this program and 
recommending to the State Board of Education those public 
schools that should receive grants under this program and 
the duration and amount of each grant pursuant to these 
Rules. 

2.01 (3) State Board of Education. 

An elected board established pursuant to Section 1, Article 
IX of the State Constitution. 

2.01 (4) Public School. 

A school that receives a majority of its funding from 
moneys raised by a general state, county, or district tax and 
whose property is owned and operated by a political 
subdivision of the state. 

2.01 (5) Levels of Literacy and Reading Comprehension Skills. 

The proficiency levels established pursuant to 2.01 (6), 

2.02 (1), (2) and (3) of these Rules and the Rules for the 
Administration of the Colorado Basic Literacy Act, adopted 
in May of 1997 by the State Board of Education in 
compliance with 22-7-501 - 505, C.R.S. 
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2.01 (6) Colorado Department of Education fCDE) - 

Request for Proposal fRFP) Development Process 
The processes and procedures developed by CDE to ensure 
that all RFP’s are consistent with the appropriate funding 
source, adhere to appropriate statutory requirements, and 
support the organizational commitment of CDE. 

2207-R-2.02 Implementation Procedures. 

2.02 (1) Application Procedures. 

CDE will be the responsible agency for implementing the 
Read to Achieve Grant Program. CDE will develop a 
request for proposal (RFP), pursuant to CDE’s RFP 
process and pursuant to the requirements and timelines 
found in 22-7-506, C.R.S. 

2.02 (2) Criteria and Levels of Reading and Literacy 
Comprehension Skills. 

The RFP to be developed by CDE must support and be 
congruent with the Rules for the Administration of the 
Colorado Basic Literacy Act adopted by the State Board of 
Education in May of 1997. The RFP shall set high, but 
attainable levels of literacy and reading comprehension 
skills for each eligible grade using the following 
assessments which are a part of the Rules for the 
Administration of the Colorado Basic Literacy Act. Both 
eligibility for initial funding and continued funding status 
will be based on levels of reading and comprehension skills 
on the following assessments: 

2.02 (2)(a) Individual Literacy Plan (ILP) status as 
defined in the RFP developed pursuant to 

2.02 (1) of these Rules. 

2.02 (2)(b) Third grade Colorado Student Assessment 
Program (CSAP) results as defined in the 
RFP developed pursuant to 2.02 (1) of these 
Rules. 
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2.02 (3) 



2.02 (4) 



O 

ERLC 



2.02 (2)(c) An individual reading assessment defined in 
the RFP developed pursuant to 2.02 (1) of 
these Rules. 

Year Two and Three Funding. 

Year two and three funding shall be contingent upon 
assessment results demonstrating that at least twenty-five 
percent of the pupils enrolled in the intensive reading 
program for the full twelve months within the prior twelve 
month period improved their reading skills to at least grade 
level or achieved proficiency on the assessments defined in 
2.02 (2) of these Rules. 

Evaluation of Program. 

The Colorado Department of Education will contract with 
an independent evaluator to conduct an annual evaluation 
of the program. Results will be disseminated to the State 
Board of Education, the Read to Achieve Board, the 
Governor, and all Colorado school districts and BOCES as 
well as other interested parties. 
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v ’’'r r Read to Achieve Board 
August 20Q0- 



Senator John Andrews 



Karen Brown, Teacher with Reading Expertise 



Pat Chlouber, State Board Member 



Laura "Gail" Coleman, Third Grade ElementaryTeacher 



Karen Durica, Literacy Coordinator, Elementary Expertise 



Darlene Medina, Third Grade Teacher, Rural District 



William J. Moloney, Commissioner of Education 



Terri Rayburn, Parent 



‘Shirley Squier, Retired Principal, Rural Elementary 



Representative Abel Tapia 



“John Youngquist, Principal, Urban Elementary 



Colorado State Senate, District 27 



Pueblo City School District 60 
(Bessemer Elementary) 



Colorado State Board of Education 
(Title I Teacher, Lake County) 



Jefferson County School District R-1 
(Bear Creek Elementary) 



Douglas County Public Schools 



Del Norte School District 



Colorado Department of Education 



Jefferson County 



Canon City 



Colorado House of Representatives, 
District 46 



Denver Public Schools 
(Newlon Elementary) 



‘Replaced by Sheryl Weitzel, Eaton School District 

“Will be replaced by new principal at Urban Elementary, name to be announced 



Attachment C 



Read to Achieve Comments 
June 2001 

Grant Proposal School 

LEA Region 



Summary of Strengths/Areas Not Addressed, Incomplete, or Insufficiently Clear 

Listed below are the expectations from Level 3 of the Read to Achieve Scoring Rubric. The boxes 
checked represent the primary areas in which readers noted areas not addressed, incomplete, or 
insufficiently clear. 

Co mponent 1: Needs Assessment 



X 


Clearly indicates reading needs of all students eligible to be served based upon a 
comprehensive analysis of disaggregated academic performance data. 


X 


Clearly addresses CBLA criteria. 


X 


Clearly indicates the professional development needs of teachers and other staff in 
the area of reading instruction based upon thorough analysis of needs assessment 
data. 



Strengths: 

♦ Needs for teacher training were recognized. 

Suggestions: 

♦ Provide disaggregated data on student reading needs based upon CBLA criteria. 

♦ Provide clear evidence of assessment of teacher needs based upon student needs 
assessment data in reading. 



C omponent 2: Goals/Evaluation of the Proposed Program 



X 


Details goals consistent with the stated purposes of the program that fully meet the 
SMART expectations. 




Includes measurable indicators of success for each goal with detail on how 
measured (measurement tools). 


X 


Clearly addresses each of the CBLA expectations regarding use of literacy 
assessment data. 




Provides for disaggregation of results within the expectations for student 
performance. 


X 


Provides credible plan for assuring valid and reliable data. 



Suggestions: 

* Develop specific goals that relate directly to identified student and staff needs. 

* Write measurable student achievement goals, based on CBLA expectations. 

* Provide for disaggregated student achievement results. 

* Provide a plan for assuring that the data you are obtaining is valid and reliable. A body of 
evidence does not assure validity and reliability. 



Component 3: Research-based Plan of Action 





Details specific action plan that fully addresses each of the four elements. 




Anchors actions within plan to strategies proven successful. 




Fully addresses implementation of allowable activities. 
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3a. Intensive Instruction 



X 


Includes convincing evidence of attention to the six dimensions of reading, 
necessary integration with regular classroom instruction, and inclusion of sufficient 
time. 




Clearly indicates rationale for choices of instruction. 



3b. Support for Teachers 



X 


Details plan with effective results-based professional development for the program. 


X 


Exemplifies Guidelines for Professional Development and fully addresses the four 
essential elements. 



3c. Parent/Family Involvement 



X 


Fully addresses provision of appropriate family literacy services. 




Indicates an exemplary plan for providing training and materials for parents. 



3d. Sustaining Impact 





Clearly addresses how impact of funds will extend beyond duration of grant. 


X 


Fully addresses effective coordination of grant activities and resources (as 
appropriate). 




Provides evidence of clear commitments of external resources (including benefits of 
consortium where appropriate). 



Strengths: 

♦ All four elements have been addressed in an effective plan for action. 

♦ Student Success Teams are a good concept. 

♦ Parents are actively involved at home in student reading interventions. 

Suggestions: 

♦ Explain how data-driven strategies will direct explicit classroom instruction in reading. 

♦ Although you mention the Guidelines for Professional Development , you do not explain 
how they will be integrated into your plan. 

♦ Identify specifically how activities of the grant will be coordinated. 



Co mponent 4: Capacity for Success 





Provides convincing evidence that the school and/or consortium is likely to succeed. 


X 


Builds a strong case for how the five elements listed above will support success of 
effort. 




Clearly addresses how successful practices and strategies derived from this program 
will be integrated into the ongoing operation of the school. 



Strengths: 



♦ All five elements are addressed. 

Suggestion: 

• Provide stronger assurance that the school is committed to the activities of this proposal. 



Co mponent 5: Cost-Effective Budget 



X 


Includes a cost-effective budget for between 1 and 3 years with appropriate narrative 
and line item descriptions for activities. 




Provides a clear picture of leveraging dollars from various funding streams to 
enhance effects of effort. 


X 


Indicates effective plan for continuation of reform efforts after funds have expired. 



Suggestions: 

• Provide more detailed information cn the supplies/materials allocation. 

• Describe how you will obtain the resources to continue the grant activities after the 



funding has ceased. 
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